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Background: Recent studies have reported that health care provided by the Veterans Health 
Administration (VHA) is at least as good as that provided in the private sector (1-2-3-4). 
These studies tended to compare a representative sample of VHA patients or hospitals with 
a representative sample not in the VHA system, after adjusting for known differences that 
might cause misleading results. Several circumstances could explain these findings. The 
VHA may provide better care than the private sector in every local area. Alternatively, non- 
VHA care may be better than VHA care in more local areas but by a small amount, whereas 
VHA care may be better than non-VHA care in fewer local areas but by a large amount in 
each area. The average across all patients and hospitals would favor the VHA in the former 
circumstance and might favor the VHA in the latter. The possibility of different explanations 
matters, because these explanations have different implications for veterans seeking health 
care. For these veterans, comparisons that provide a national average may be less useful 
than a local comparison. For example, individual veterans would not want to know whether 
VHA or non-VHA hospitals provide better care on average but whether the nearest VHA 
hospital or the local non-VHA hospital is better for them. 

Objective: To determine whether several important outcomes of VHA and non-VHA care in 
the same region differ. 

Methods and Findings: We identified 15 outcome measures that were reported by VHA and 
non-VHA hospitals by using data from the Hospital Compare Web site 
( https://data.medicare.gov/data/archives/hospital-comparej (Table 1). These measures 
included 30-day risk-adjusted mortality rates for 4 common diseases plus 11 additional 
patient safety indicators. We used each hospital's ZIP code to assign the hospital to 1 of 306 
hospital referral regions. The Dartmouth Atlas of Health Care defines these regions as 
distinct health care markets. We limited our analyses to the 121 regions in which at least 1 
VHA and 1 non-VHA hospital reported at least 1 of the measures. Dartmouth College has 
determined that analyses of publicly available data from Hospital Compare are not 
considered human subjects research (Dartmouth College Committee for the Protection of 
Human Subject number CPHS00028121). 

For each measure and region, we determined whether the score of a VHA hospital was the 
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best, above average, or the worst among all hospitals in the region. We used the t test 
(with equal variances not assumed) to compare mean VHA scores with mean non-VHA 
scores for each outcome in all regions. When we considered all regions together, VHA care 
was significantly betterthan non-VHA care for 14 of the 15 measures (Table 2), which 
confirms the findings of previous studies (1-2-3-4). A VHA hospital also provided the best 
care in at least 50% of the regions for 9 of the 15 measures, above-average care in at least 
67% of the regions for 14 measures, and the worst care in fewer than 15% of the regions 
for 14 measures. 

Discussion: We sought to use Hospital Compare data as a veteran contemplating where to 
obtain the best local care would. We found that VHA hospitals provided the best care in 
most hospital referral regions and rarely provided the worst care. This finding may indicate 
that the VHA generally provides truly excellent care; if so, we believe that efforts to 
outsource VHA care to non-VHA settings solely for patient convenience should be 
reconsidered. 

However, VHA and non-VHA hospitals may report data differently to Hospital Compare. We 
found that several of the VHA's aggregate mortality and patient safety results were 
markedly betterthan those for non-VHA hospitals. Alternatively, analyses that use patient- 
level data have found these measures to be similar to those that we found in non-VHA 
hospitals, particularly for acute myocardial infarction and heart failure mortality (3, 5). 
These findings support the possibility of differential reporting. If so, we believe that the VHA 
and Centers for Medicare & Medicaid Services should take steps to ensure that the methods 
they use to calculate Hospital Compare data present fair comparisons to end users who are 
trying to make informed health care decisions. 

Limitations of this study include its cross-sectional nature and the use of hospital referral 
regions from The Dartmouth Atlas of Health Care to define health care markets and limited 
data elements. As such, comparisons using different periods, market definitions, and data 
elements may produce different results. 


William B. Weeks, MD, PhD, MBA 

The Dartmouth Institute for Health Policy and Clinical Practice; Lebanon, New Hampshire 

Alan N. West, PhD 

White River Junction VA Medical Center; White River Junction, Vermont 


Note: Drs. Weeks and West were previously employed by the Veterans Health Administration. Some data used in 
this analysis were obtained from the The Dartmouth Atlas of Health Care, which is funded by the Robert Wood 
Johnson Foundation and the Dartmouth Clinical andTranslational Science Institute under award UL1TR001086 
from the National Center for Advancing Translational Sciences of the National Institutes of Health. 

Disclosures: Disclosures can be viewed at 

www.acponline.orq/authors/icmie/ConflictOfInterestForms.do?msNum=M18-154Q . 

Reproducible Research Statement: Study protocol: See the Supplement (available at Annals.org). Statistical 
code: SPSS files (IBM) are available from Dr. Weeks (e-mail, wbwtgida rtmouth.edu ). Data set: SeeTable 1. 







EMBARGOED FOR RELEASE UNTIL 5:00 PM ET ON MONDAY, DECEMBER 10, 2018 

References 

1. O'Hanlon C, Huang C, Sloss E, Anhang Price R, Hussey P, Farmer C, et al. Comparing VA and non-VA quality of 

care: a systematic review. J Gen Intern Med. 2017;32:105-21. [PMID: 27422615] doi: 10.1007/sll606-016- 
3775-2 

2. Blay E Jr, DeLancey JO, Hewitt DB, Chung JW, Bilimoria KY. Initial public reporting of quality at Veterans Affairs 

vs non-Veterans Affairs hospitals. JAMA Intern Med. 2017;177:882-5. [PMID: 28418527] 
doi: 10.1001/jamainternmed.2017.0605 

3. Nuti SV, Qin L, Rumsfeld JS, Ross JS, Masoudi FA, Normand SL, et al. Association of admission to Veterans 

Affairs hospitals vs non-Veterans Affairs hospitals with mortality and readmission rates among older men 
hospitalized with acute myocardial infarction, heart failure, or pneumonia. JAMA. 2016;315:582-92. [PMID: 
26864412] doi: 10.1001/jama.2016.0278 

4. Anhang Price R, Sloss EM, Cefalu M, Farmer CM, Hussey PS. Comparing quality ofcare in Veterans Affairs and 

non-Veterans Affairs settings. J Gen Intern Med. 2018;33:1631-8. [PMID: 29696561] doi: 10.1007/sll606- 
018-4433-7 

5. Rivard PE, Elixhauser A, Christiansen CL, Shibei Zhao, Rosen AK. Testing the association between patient safety 

indicators and hospital structural characteristics in VA and nonfederal hospitals. Med Care Res Rev. 
2010;67:321-41. [PMID: 19880671] doi: 10.1177/1077558709347378 


Table 1. Data Sources and Time Spans Covered* 

Table 2. Measures of VHA Care 

Supplement. Protocol for VA to Non-VA Hospital Performance Comparison Using 
Hospital Compare 



EMBARGOED FOR RELEASE UNTIL 5:00 PM ET ON MONDAY, DECEMBER 10, 2018 


Table 1. Data Sources and Time Spans Covered* 


Variable 

30-d RAMRs for acute myocardial infarction, COPD, heart 
failure, or pneumonia 

VHA 

Non-VHA 

Reporting period 

7/1/14-6/30/17 

7/1/14-6/30/17 

Files used| 

Readmissions and 
Deaths - 

VA_07_10_2018.csv 
and Readmissions 
and Deaths - COPD 

- VA.csv 

Complications and 
Deaths - 
Hospital.csv 

PSI 3, 4, 6, 8, 9, 10, 11, 12, 13, 14, and 15* 

Reporting period 

10/1/15-6/30/17 

10/1/15-6/30/17 

Files used for PSI analysis! 

VAPS l_07_10_18. c 

sv 

Complications and 
Deaths - 
Hospital.csv 

Data source for the crosswalk between ZIP code and HRR 

ZipHsaHrr16.xls, 
accessed at 
www.dartmouth 
atlas.org/tools/d 
ownloads. aspx?t 
ab=39#zip_cros 
swalks under 

ZIP code 
crosswalks, 

2016 

ZipHsaHrr16.xls, 
accessed at 
www.dartmouth 
atlas.org/tools/d 
ownloads. aspx?t 
ab=39#zip_cros 
swalks under 

ZIP code 
crosswalks, 

2016 

Hospitals reporting >1 measure, n 

135 

2988 

Range of hospitals reporting in each HRR, n 

1-2 

1-78 

HRR-defined health care markets represented, n 

121 

121 


COPD = chronic obstructive pulmonary disease; HRR = hospital referral region; PSI = patient safety 
indicator; RAMR = risk-adjusted mortality rate; VHA = Veterans Health Administration. 

* Data are for VHA and non-VHA hospitals in the 50 states and the District of Columbia that reported to 
the Centers for Medicare & Medicaid Service’s Hospital Compare Web site and were in HRR-defined 
health care markets that included >1 VHA and 1 non-VHA reporting hospital, 
t Available from the zipped 2018 HOSArchive_Revised_FlatFiles folder at 

https://data.medicare.qov/data/hosoital-compare . 

! See Table 2 for an explanation of PSIs. 






Table 2. Measures of VHA Care 


Measure Using Descriptions Provided in Hospital HRRs in HRRs According to a VHA Mean Aggregated Pfor 

Compare Which >1 VHA Hospital’s Performance Score, Performance Scores for Difference 

and 1 Non- % All 121 Markets (SD) * 

VHA Hospital 
Provided Data, 
n 


30-d RAMRs 


Best 

Above 

Average 

Worst 

VHA 

Non-VHA 


Acute myocardial infarction 

83 

15.7 

74.7 

2.4 

12.48 (1.05) 

13.14 (1.22) 

<0.001 

COPD 

116 

94.0 

100 

0 

4.89 (0.91) 

8.35 (1.11) 

<0.001 

Heart failure 

116 

42.2 

91.4 

0.9 

10.04 (1.59) 

11.71 (1.69) 

<0.001 

Pneumonia 

116 

25.0 

84.5 

2.6 

13.93 (1.67) 

15.75 (1.98) 

<0.001 


Other PSIs 


Pressure sores (PSI 3)| 

116 

43.1 

57.8 

13.8 

0.474 (0.60) 

0.399 (0.47) 

0.165 

Death among patients with serious treatable complications 
after surgery (PSI 4)t 

65 

76.9 

83.1 

10.8 

104.5 (57.9) 

161.7 (16.8) 

<0.001 

Collapsed lung due to medical treatment (PSI 6) 

116 

71.6 

77.6 

14.7 

0.137 (0.21) 

0.288 (0.05) 

<0.001 

Broken hip from a fall after surgery (PSI 8)f 

116 

74.1 

75.0 

26.7 

0.081 (0.17) 

0.111 (0.01) 

0.047 

Perioperative hemorrhage or hematoma rate (PSI 9)f 

101 

61.4 

77.2 

13.9 

1.64 (1.8) 

2.58 (0.35) 

<0.001 

Postoperative acute kidney injury requiring dialysis (PSI 

10)t 

95 

67.4 

77.9 

13.7 

0.800 (1.35) 

1.31 (0.30) 

<0.001 

Postoperative respiratory failure (PSI 11) 

95 

48.4 

78.9 

8.4 

5.96 (7.01) 

8.01 (2.87) 

0.005 

Serious blood clots after surgery (PSI 12)f 

101 

47.5 

70.3 

13.9 

3.05 (2.4) 

3.84 (1.0) 

0.001 

Bloodstream infection after surgery (PSI 13) 

95 

62.1 

75.8 

10.5 

3.69 (4.1) 

5.22 (1.1) 

<0.001 

A wound that splits open after surgery on the abdomen or 

97 

87.6 

87.6 

10.3 

0.469 (1.91) 

0.856 (0.24) 

0.043 










pelvis (PSI 14) 

Accidental cuts and tears from medical treatment (PSI 15)t 108 86,1 95 ' 4 3,7 0.37(0.75) 1.29(0.23) 


COPD = chronic obstructive pulmonary disease; HRR = hospital referral region; PSI = patient safety indicator; RAMR = risk-adjusted mortality 
rate; VHA = Veterans Health Administration. 


* Equal \rariances were not assumed. 

f In >1 HRR in which 2 VHA hospitals were reporting, 1 VHA hospital performed the best in the HRR and the other performed the worst. 


< 0.001 





